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ABSTRACT 

Designs for sequential sampling procedures that adapt 
to cumulative information are discussed. A familiar illustration is 
the play-the-winner rule in which there are two treatments; after a 
random start, the same treatment is continued as long as each 
successive subject registers a success. When a failure occurs, the 
other treatment is used until failure occurs. Sequential sampling 
procedures are discussed in relation to clinical trials, but there 
are many applications of such procedures in statistics. Models are 
presented for the adaptive allocation of dose levels to subjects 
arriving sequentially. Natural multivariate design spaces occurring 
in radiotherapy given at the Fred Hutchinson Cancer Research Center 
motivate this work. The allocation rules proposed adapt to 
accumulating failure data, possibly censored, and thus are governed 
by an underlying, unknown, multivariate failure distribution in the 
limit. An urn model is used to effect the adaptation of the 
allocation scheme. A recursive algorithm for updating the urn 
distribution is derived. The Bernoulli environment and urn models on 
ordinal sample spaces in a random environment are also considered. 
(SLD) 
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ABSTRACT 

Models are presented for the adaptive 
allocation of dose levels to subjects arriving 
sequentially. Natural multivariate design spaces 
occurring in radio-therapy given at the Fred 
Hutchinson Cancer Research Center motivate 
this work. The allocation rules proposed adapt 
to accumulating failure data, possibly censored, 
and thus are governed by an underlying, 
unknown, multivariate failure distribution in the 
limit. An urn model is used to effect the 
adaptacion of the allocation sch me. A recursive 
algorithm for updating the urn distribution is 
derived. 

L BACKGROUND 

This paper is concerned with designs for 
sequential sampling procedures that adapt go 
cumulative information. A familiar illustration 
is the play-the-winner rule in which there are 
two treatments; after a random start, the same 
treatment is continued as long as each successive 
subject registers a success. When a failure 
occurs, the other treatment is used until another 
failure is registered. This allocation strategy is 
continued until a predetermined stopping criter- 
ion is met, at which time the decision as lo 
which treatment is "better*^ is made by 
probabilistic criteria. This design, and many 
variants thereof, have been the source of 
numerous mathematical analyses such as 
described in Chow, Robbins and Siegmund 
(1971). Sequential analysis has become a large 
subfield in statistics and we briefly sketch the 
further development of sequential methods as it 
has motivated this work. 

Herein, the term subject refers to a generic 
experimental unit. Exchanging the term subject 
to component or cells would connote industrial 
or laboratory applications, respectively. It is 
important to note that the designs depend on 
the spatial and temporal features cf the measure- 
ments being observed on each experimental unit. 



The designs considered will be especially useful 
when the experimental units are expensive. 

Sequeniml samplusg procedures were 
developed during World War II to analyze 
observations taken in sequence. This was 
particularly suitable for ir/ ustrial inspections in 
which measurements can be taken sequentially 
from objects arriving on conveyor belts. These 
early developments assume the outcome measure- 
ment is taken on each subject before another 
subject arrives. Armitage (1960) presents a 
review of sequential clinical experiments. Hoel, 
Sobcl and Weiss (1975) survey adaptive 
sampling methods for clinical trials and report 
that, to the best of their knowledge at that time, 
no sequential methods were being used in clinical 
experiments. Two major bariiers to the applica- 
tion of sequential methods to clinical studies at 
that time were that most clinical results are not 
observed in the same sequence as subjects are 
treated) and that observations may be truncated 
(censored). 

Two theoretical thrusts have led to the 
widespread use of sequential analytic methods 
when observations are truncated. First, the 
development of nonsequential methods for 
truncated observations had to precede the 
development of sequential designs for such 
observations. Statistics comparing two treat- 
ments were first developed by Gehan (1965) and 
Mantel (1967), and extended, by Breslow (1970) 
to accommodate k treatments. By the time the 
two papers by Peto, Pike, Armitage, Breslow, 
Cox, Howard, Mantel, McPherson, Peto, and 
Smith (1976, 1977) codified the now classical 
design for clinical trials, a fixed two point 
sampling space already was rapidly becoming a 
design standard when failure distributions were 
positive for years after a subjects arrives and is 
treated. This standard two point sampling 
space refiected the forefront of the existing 
statistical tests permitting truncated realizations 
of the failure process. From 1975 to 1986, we 
designed and conducted dozens of such 
experiments at the Fred Hutchinson Cencer 
Research Center atid other medicaL institutions 
did likewise. 

The value of sequential decision rules for the 
treatment of patients was apparent, which led to 



a ma^or research thrust that centered around the 
development of sequential tests for two point 
sampling spaces. Because of logistic difficulties 
in the retrieval and management of data derived 
from monitoring subjects tu obtain failure times, 
group sequential approaches became practical in 
many applications where fully sequential designs 
were not. Therefore, this research thrust focused 
on the development of group sequential methods 
(sec Pocock (1977), Harrington, Flemming, and 
Green (1982), Whitehead (1983) and Fleming, 
Harrington, and O'Brien (1984)). An adaptive 
multistage rule was proposed by Tsutakawa 
(1972) for observations that are logistically 
distributed. More recently, increased attention 
is being directed at multistage procedures. For 
example, see Hall (1981), Cohen and Sackrowitz 
(1984), Witmer (1986), Clayton and Witmer 
(1988) and Lorden (1988). 2. 

About the time that sequential methods for 
comparing two truncated failure processes were 
being developed, attention was focused on 
studies with more that two treatments. Wamp- 
ler, Carter, and Williams (1978) and Stablein, 
Carter, and Wampler (1980) studied exper- 
imental designs for estimating parameters from 
truncated sample failure processes on multidi- 
mensional sample spaces. But these have not 
been extended to models dealing wit'a the 
sequential arrival of subjects. Flournoy (1P86) 
proposed combining group sequential arrival 
models with a sequential stopping rule where 
multiple treatments could be ranked on an 
ordinal scale, and experimental designs based on 
this model are now being applied at the Fred 
Hutchinson Cancer Research Center. These 
models contain a mix of theoretical and heuristic 
considerations, including aspects of the se- 
quential allocation rules proposed by Dixon 
(1965) and Wetherill (1966) with the two-stage 
Bayesian designs developed by Tsutakawa 
(1972). The need to formalize the key objectives 
of these applications motivate this paper. 

The sampling designs described are directed 
at the problem of estimating the relative effects 
of treatment on the underlying failure distri- 
bution. They are based on a modification and 
extension of Polya-Eggenberger urn models. 
This idea arose as an extension of the play-the- 
winner rule by Zelen (1969), from a design for 
an outcome that has a Bernoulli distribution, to 
one in which the outcome is distributed as a 
general time varying failure process In these 
models, a treatment decision for an arriving sub- 



ject must be made before the outcome of the 
previous trial is observed. Zelen described how 
urn models could be used to decouple arrival and 
failure sequences. Wei (1979, 1988) characteriz- 
ed this model mathematically and developed sta- 
tistical tests based on it. 

In Section 2, we describe an extended Polya- 
Eggenberger urn model with a generalized 
sample space in the environment of a failure 
process. In Section 3, we exemplify this 
generalized urn model by assuming the failure 
process has a conditional Bernoulli distribution 
and, in Section 4, a specific replacement strategy 
is incorporated into the model. Finally, in 
Section 5, a recursive algorithm for the evolving 
urn distribution is developed in terms of the 
observed failures and the replacement algorithm. 

URN MODELS ON ORDINAL SAMPLE SPACES 
IN A RANDOM ENVIRONMENT 

The class of P6lya*Eggenberger urn models 
was designed to model potential contagion (for a 
general description, see Johnson and Kotz 
(1977)). A ball is taken from an urn with b 
black and r red balls, its color is noted, and that 
ball plus s b i of the same color are returned to 
the urn. e that Po1y a- Eggenberger urn 
models forw. a general class in which balls are 
distinguished by a categorical feature, such as 
color. Hill, Lane and Sudderth (1980) general- 
ized the Polya-Eggenberger urn models, 
retaining the categorical labeling of balls, while 
studying the asymptotic properties of several sto- 
chastic replacement strategies. Chen and Sta;r 
(1980) studied optimal stopping wh^n drawing 
balls with ordinal labels from an urn, but their 
definition of ^optimal' reflects a different goal 
from the one described here. 

We extend the Polya-Eggenberger models in 
two central ways: we assume (1) a random 
process governs not only the draving of balls, 
but their replacement as well, and (2) balls are 
categorized according to an ordinal attribute 
that maps to a parameter in the distribution 
governing the replacement of balls. The first 
feature places the urn ,nodel in a ^random 
environment'; the second links the drawing and 
replacement distributions by means of a bivari- 
ate time series so that the drawing distribution 
roay adapt to the characteristics of the 
replacement distribution. The replacement 
strategy is specified as a mapping that is a 
function of the experimental resultiS. This map 



links the urn distribution to the failure dictri- 
bution. 

Consider a sequence of trials, in each of 
which a subject receives a treatment that 
involves a particular level or dose. The level 
that will be most effective is unknown, and it 
may be that a single optimal treatment level 
does not exist* Instead, it is expected that 
efficacy follows a probability distribution over 
the treatment space. Indeed, the failure density 
may well be composed of a mixture in the sense 
that one failure process may predominate if the 
treatment level is too low and another may 
predominate if the treatment level is too high. 
If a treatment level is too low, then we want to 
increase the level for subsequent subjects. Also, 
the treatment may be toxic or expensive, so we 
want to decrease the level for subsequent sub- 
jects if it is higher than necessary. 

3- SAMPLING FROM AN ORDINAL URN 
PROCESS 

For each subject, a ball is drawn that 
determines the treatment level. We construct a 
sampling protocol that adapts to the distribution 
of the minimum effective treatment level. This 
is accomplished by letting the unknown random 
failure proces3(ts) that governs the outcome of 
treatment also govern the replacement of balls in 
the u'^n. 

Consider an urn with balls labeled j=:l, ... , 
J (J large) where the label determines the treat- 
ment level. The number of balls in the urn at 
each level is a randon. variable that changes as 
outcomes are observed. Let /?j^=(/?j^^, ... , p^j) 
be the random vector whose j-th element, is 
the number of balls at level j just prior to the 
time that the k-th ball is drawn. Let assume 
vector values b|^ with elements 

\ e {O. 1.2 E by= Bk}. 

Then P{j^j^ =: bj^} denotes the urn distribution 
upon the arrival of the k-th subject. The initial 
urn distribution and the total starting number of 
balls in the urn are fixed, that is 

P{^o = bo: .E bojrr Bo) = 1. 

We restrict this analysis to the case when Sjby= 
B for all k which implies that the k-th ball 
drawn must be replaced before the (k+l)st draw. 



This means that failures are observed in the 
same sequence as subjects arrive for treatment 
assignments. By permitting to vary we 
would accommodate more general failure distri- 
butions than the one being studied. 

The urn distribution depends on the density 
governing the draw of the balls and the replace- 
ment. The drawing of balls conditional on the 
urn distribution as each subject arrives has a 
multinomial distribution. This portion of the 
model remains constant across the entire general 
class of models we discuss while the algorithms 
for replacing balls and the failure distributions 
will vary. 

Let the halls be drawn at random and 
consider a specific urn distribution b^. upon the 
arrival of the k-th subject. Let X^. = (X^.^, 
^k(J-i)) "^j* random vector with elements Xj^j. 
Also let ]Cj_jX|^ = li where 

Let Xy=l if a j-level ball is drawn upon the 
arrival of the k-th subject and Xj^ = 0 for all 
iT^jforjrrl, J. Letp,^ = {PkH 1 Pkj) b« 
a vector with elements 

Pkj ^ *^kj/Bi j=l, ..M J 

where J^^^^p^; = 1 for all k. 

Then conditional on the urn distribution 
^k=b|^f ¥k ^ multinomial distribution with 
parameter pj^. The event that a ball labeled 
with the j-th treatment level is chosen is 
{Xj = 1}. This event is equivalent to the vector 
valued o'^ent 

{^k = . = 1 & Xj,. = 0, V i 9^ j, i = 1, J}. 

It follows that Xj is conditionally distributed as 
a Bernoulli with parameter py and the 
conditional probability of the k-th ball being 
drawn having label j is written as: 

P{Xm = i iPk) = py (1) 

If bj^ is the (IxJ) vector tabulating the 
number of balls at <*ach level in the urn 
immediately after the k-th draw, and before any 
additional balls are placed in the urn, then 

Xk- (2) 



4. THE BERNOULLI ENVIRONMENT 

The law governing the drawing of balls from 
the urn upon arrival of the (k+l)st subject not 
only depends on the history of the experiment 
through tue urn's distribution at the k-th 
arrival, but also on realizations of the failure 
process during this inter-arrival time* Let 
denote the random failure process for the k-th 
subject. If the failure process is a function of 
time since arrival (time on trial), then naturally 
the replacement operator also would be a 
function of time since arrival. To deflne a speci- 
fic urn process in the environment of a random 
failure process, select the failure distribution and 
the replacement operator. 

At this stage of our development, we 
consider a simple failure process: 

U _J 1 if the k-th outcome is a failure 
^ |0 otherwise. 

Ms^ assume that the subjects k=l, 2, ... are 
homogeneous; and that the processes 
determining the subjects' arrival times are 
independent of each other and independent of 
the failure process. The outcome Uj^ is not 
observed unconditionally, but only conditional 
given a treatment level j. Our concern is with 
the relationship between the conditional failure 
probabilities as the treatment level varies. Let 

P{Ufc = 1 1 X^: Xy=l } = TT. 

denote the probabilities of failure given 
treatment level j and let 

P{Uk = 0 I X^: Xy=l } = l-TTj = 

j= 1, J and k=l, 2, . 

Conditional on the k-th draw, the failure process 
U]^ has a Bernoulli distribution: 

P{Uk = Uk I X^: X^=l } = Trjk f »k, (3) 

j=: 1, J and kr=l, 2, ... , 

where we write.^ = 1 — z. 

From equation (3), it is clear that once the 
treatment level is determined by the draw Xj^, 
the failure process Uj^ has no further dependence 
on the urxi distribution or on its prior evolution. 
When equations (1) and (3) are applied to the 



joint probability mass function of the treatment 
and outcome for subject k conditional on the urn 
distribution ^|^, the joint probability factors 
into two terms, only one of which depends on 
the urn distribution: That is, when {X|^=l}, we 
have 

n\ = Ufc, X^ = Xk I p^} (4) 

= P{Uk = Uk I X^: X^j=l } P{Xy = 1 I p,} 

= 7rjk7r.kpy . 

Let y|^ be the number of failures among the 
first k subjects treated at the j-th dose level and 
n|^- the number of trials at the j-th dose leveh 
Then 

Ylg = E uj xj. and n^j = g xy, (5) 

j = l, .., J. 

The difference "y-yw is the number of 
successes among the first k trials. The condition- 
al distribution of yj^j given nj^. is Binomia^ (nj^., 
TTj). Furthermore, the k-th subject's marginal 
failure density, conditional on /?|^ is 

P{Uk = Ukl^k = PkB} 
= i: Trjk TTj'k p^. 

Let CU|^, 95J be a (kx(J f 1)) matrix in 
which the m-th row is (11,,^, X^^^). Then the 
total accumulation of random variables through 
the k-th trial is contained in (Hlj^, 9Bj^). 
Analogously, let be the (kx(J + l)) matrix of 
observations accumulated through the k-th trial: 
That is, the m-th row of («, is (Uj^^, x^) for 
m - 1, k. 

Then using Equation (4), the density of 
(^j^,95^.) conditional on the evolving urn distribu- 
tion parameters {p^, p,^} factors into a 
product of densities: 

p{{\, = Jk) I Pi Pk} 

= n P{Um = "m I X J P{X^ = X„ IpJ. 
m=:l 

Note that only the second term in the product 
depends on the history of the urn distribution. 
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5. AN EXEMPLARY REPLACEMENT 
STRATEGY 

For an initial replacement strategy, let 
B)g=B for all k and define the replacement 
operator to restrict the level on the replacement 
ball to be adjacent to the level on the ball 
drawn. If the j-th Lall is drawn upon the arrival 
of the k-th subject, then one such strategy is 
defined by the following recursive formula: 

^k+i)j = + (Xk(j.i)Uk + ^kO+l)"k)» (6) 
where the subscripts are taken mod(J) so that 



and 



^k+i)J = ^kJ + (xj.iUk + XiU,,). 



The cyclic behavior is adopted in order to make 
the transformation one to one. In the applica^ 
tions motivating this work, the number and 
range of treatment levels is sufficiently large that 
the replacement process should reach the 
boundaries with null probability. In other 
words, the treatment will be 'successful' at the 
highest levels, and J, and may well be far 
above the minimally successful level with severe 
toxicities. Similarly, at the lowest levels the 
treatment is knoM n to be inefTective (the lowest 
level may be no treatment at all). If these condi- 
tions hold, the replacement process will never 
actually cycle. 

The strategy expressed in (6) is just one of a 
wide class of interesting alternative replacement 
strategies, 'ut it's simplicity is useful to 
illustrate i. general class of adaptive models 
while permit! »g a number of their characteris- 
tics to be developed. Other strategies will be 
needed for varying underlying shapes of the 
dose-response surface. This strategy has the 
following interpretation: If U=l there has been 
a failure and consequently a ball labeled with 
the next higher level is added to the urn, 
whereas if U=0, a ball marked with th». next 
lower level is added to the urn. 

We now extend (6) to a matrix repre- 
sentation. Let R,^ = R(ui^) be the JxJ matrix 
operating the replacement strategy. If the 
failure process wa« more general than the 
Bernoulli, Rj^ would be a function the multiple 



outcomes under observation at the time of the k- 
th arrival, and thus a function of both the 
failure times and the interarrival times. 

Let Circ(ci, cj) denote a circulant 
matrix: 



Circ(ci, cj) = 



Cj C2 



Cg - Cj Cj 



In the example described by (6), the replacement 
operator is the circulant matrix 

R,, = Circ(0, u, 0, 0, u). 

Therefore the change in the distribution of balls 
in the urn induced by the k-th subject can be 
written as 



bk+i" =5k(R'k-I)i 



(7) 



where subtracting the identity matrix accounts 
for the influence of the draw from the urn as 
given in (2). 

6. ADAPTING THE URN PROCESS 

To find the distribution of the urn as it 
evolves, we first decompose the replacement 
operator. Then we transform the joint density 
of the eccumulating dra vs and the experimeriLal 
results to a density of partial sums. 
Conditioning on one of the partial sums, we 
derive the urn distribution. 

Define JxJ dimensional circulant matrix 
P = Circ(0, 0, 1), for which 

P-I = pT = Circ(0, 1, 0, 0), 

where P"^ is the inverse and P^ is the transpose 
of P. Note that P operating on x shifts the 
elements in x to the next higher dose level, 
whereas P^ shifts the elements in x to the next 
lower dose level. Let Rj^= Uk^+^kP^- 
using the difTerence equation (7), the urn distri- 
bution in (6) can be expressed in terms of the 
partial sums defined in (5): 



i=:l 



(8) 



= nk(P -I) H- yk(P^~P). 
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Thus we have derived an general algorithm 
by which the urn distribution can be updated. 
When the urn distribution reaches a steady 
state, the influence of the initial urn distribution 
will be negligible and it will reflect the failure 
distribution. A more precise characterization of 
this evolving reflection is currently under study. 
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